Spectral Clustering with Two Views

In this paper we develop an algorithm for spectral clustering in the multi-view setting where there are two independent subsets of dimensions, each of which could be used for clustering (or classification). The canonical examples of this are simultaneous input from two sensory modalitites, where input from each sensory modality is considered a view, as well as web pages where the text on the page is considered one view and text on links to the page another view. Our spectral clustering algorithm creates a bipartite graph and is based on the “minimizing-disagreement” idea. We show a simple artifically generated problem to illustrate when we expect it to perform well and then apply it to a web page clustering problem. We show that it performs better than clustering in the joint space and clustering in the individual spaces when some patterns have both views and others have just one view. Spectral clustering is a very successful idea for clustering patterns. The idea is to form a pairwise affinity matrix A between all pairs of patterns, normalize it, and compute eigenvectors of this normalized affinity matrix (graph Laplacian)L. It can be shown that the second eigenvector of the normalized graph Laplacian is a relaxation of a binary vector solution that minimizes the normalized cut on a graph (Shi & Malik, 1998; J.Shi & Malik, ; Meila & Shi, 2001; Ng et al., 2001). Spectral clustering has the advantage of performing well with non-Gaussian clusters as well as being easily implementable. It is also non-iterative with no local minima. The Ng,Jordan,Weiss(Ng et al., Appearing in Proceedings of the Workshop on Learning with Multiple Views, 22 ICML, Bonn, Germany, 2005. Copyright 2005 by the author(s)/owner(s). 2001) (NJW) generalization to multiclass clustering (which we will build on) is summarized below for data patterns xi to be clustered in to k clusters. • Form the affinity matrix A(i, j) = exp(−||xi − xj ||/2σ) • Set the diagonal entries A(i, i) = 0 • Compute the normalized graph Laplacian as L = D−.5AD−.5 where D is a diagonal matrix with D(i, i) = ∑ j A(i, j) • Compute top k eigenvectors of L and place as colums in a matrix X • Form Y from X by normalizing the rows of X • Run kmeans to cluster the row vectors of Y • pattern xi is assigned to cluster α iff row i of Y is assigned to cluster α In this paper we develop an algorithm for spectral clustering in the multi-view setting where there are two independent subsets of dimensions, each of which could be used for clustering (or classification). The canonical examples of this are multi-sensory input from two modalities where input from each sensory modality is considered a view as well as web pages where the text on the page is considered one view and text on links to the page another view. Also computer vision applications with multiple conditionally independent sensor or feature vectors can be viewed in this way. 1. Algorithm Development Our spectral multi-view algorithm is based on ideas originally developed for the (non-spectral) MinimizingDisagreement algorithm (de Sa, 1994a; de Sa & Ballard, 1998). The idea behind the MinimizingDisagreement (M-D) algorithm is that two (or more)